Causal Reinforcement Learning using Observational and Interventional Data

Learning efficiently a causal model of the environment is a key challenge of model-based RL agents operating in POMDPs. We consider here a scenario where the learning agent has the ability to collect online experiences through direct interactions with the environment (interventional data), but has also access to a large collection of offline experiences, obtained by observing another agent interacting with the environment (observational data). A key ingredient, that makes this situation non-trivial, is that we allow the observed agent to interact with the environment based on hidden information, which is not observed by the learning agent. We then ask the following questions: can the online and offline experiences be safely combined for learning a causal model ? And can we expect the offline experiences to improve the agent's performances ? To answer these questions, we import ideas from the well-established causal framework of do-calculus, and we express model-based reinforcement learning as a causal inference problem. Then, we propose a general yet simple methodology for leveraging offline data during learning. In a nutshell, the method relies on learning a latent-based causal transition model that explains both the interventional and observational regimes, and then using the recovered latent variable to infer the standard POMDP transition model via deconfounding. We prove our method is correct and efficient in the sense that it attains better generalization guarantees due to the offline data (in the asymptotic case), and we illustrate its effectiveness empirically on synthetic toy problems. Our contribution aims at bridging the gap between the fields of reinforcement learning and causality

The Kepler-10 planetary system revisited by HARPS-N: A hot rocky world and a solid Neptune-mass planet

Kepler-10b was the first rocky planet detected by the Kepler satellite and con- firmed with radial velocity follow-up observations from Keck-HIRES. The mass of the planet was measured with a precision of around 30%, which was insufficient to constrain models of its internal structure and composition in detail. In addition to Kepler-10b, a second planet transiting the same star with a period of 45 days was sta- tistically validated, but the radial velocities were only good enough to set an upper limit of 20 Mearth for the mass of Kepler-10c. To improve the precision on the mass for planet b, the HARPS-N Collaboration decided to observe Kepler-10 intensively with the HARPS-N spectrograph on the Telescopio Nazionale Galileo on La Palma. In to- tal, 148 high-quality radial-velocity measurements were obtained over two observing seasons. These new data allow us to improve the precision of the mass determina- tion for Kepler-10b to 15%. With a mass of 3.33 +/- 0.49 Mearth and an updated radius of 1.47 +0.03 -0.02 Rearth, Kepler-10b has a density of 5.8 +/- 0.8 g cm-3, very close to the value -0.02 predicted by models with the same internal structure and composition as the Earth. We were also able to determine a mass for the 45-day period planet Kepler-10c, with an even better precision of 11%. With a mass of 17.2 +/- 1.9 Mearth and radius of 2.35 +0.09 -0.04 Rearth, -0.04 Kepler-10c has a density of 7.1 +/- 1.0 g cm-3. Kepler-10c appears to be the first strong evidence of a class of more massive solid planets with longer orbital periods.Comment: 44 pages, 8 figures, accepted for publication in Ap

Precise Masses in the WASP-47 System

We present precise radial velocity observations of WASP-47, a star known to host a hot Jupiter, a distant Jovian companion, and, uniquely, two additional transiting planets in short-period orbits: a super-Earth in a ~19 hour orbit, and a Neptune in a ~9 day orbit. We analyze our observations from the HARPS-N spectrograph along with previously published data to measure the most precise planet masses yet for this system. When combined with new stellar parameters and reanalyzed transit photometry, our mass measurements place strong constraints on the compositions of the two small planets. We find unlike most other ultra-short-period planets, the inner planet, WASP-47 e, has a mass (6.83 +/- 0.66 Me) and radius (1.810 +/- 0.027 Re) inconsistent with an Earth-like composition. Instead, WASP-47 e likely has a volatile-rich envelope surrounding an Earth-like core and mantle. We also perform a dynamical analysis to constrain the orbital inclination of WASP-47 c, the outer Jovian planet. This planet likely orbits close to the plane of the inner three planets, suggesting a quiet dynamical history for the system. Our dynamical constraints also imply that WASP-47 c is much more likely to transit than a geometric calculation would suggest. We calculate a transit probability for WASP-47 c of about 10%, more than an order of magnitude larger than the geometric transit probability of 0.6%.Comment: 15 pages, 3 figures, 3 tables. Accepted in A

The Mass of Kepler-93b and The Composition of Terrestrial Planets

Kepler-93b is a 1.478 +/- 0.019 Earth radius planet with a 4.7 day period around a bright (V=10.2), astroseismically-characterized host star with a mass of 0.911+/-0.033 solar masses and a radius of 0.919+/-0.011 solar radii. Based on 86 radial velocity observations obtained with the HARPS-N spectrograph on the Telescopio Nazionale Galileo and 32 archival Keck/HIRES observations, we present a precise mass estimate of 4.02+/-0.68 Earth masses. The corresponding high density of 6.88+/-1.18 g/cc is consistent with a rocky composition of primarily iron and magnesium silicate. We compare Kepler-93b to other dense planets with well-constrained parameters and find that between 1-6 Earth masses, all dense planets including the Earth and Venus are well-described by the same fixed ratio of iron to magnesium silicate. There are as of yet no examples of such planets with masses > 6 Earth masses: All known planets in this mass regime have lower densities requiring significant fractions of volatiles or H/He gas. We also constrain the mass and period of the outer companion in the Kepler-93 system from the long-term radial velocity trend and archival adaptive optics images. As the sample of dense planets with well-constrained masses and radii continues to grow, we will be able to test whether the fixed compositional model found for the seven dense planets considered in this paper extends to the full population of 1-6 Earth mass planets.Comment: 8 pages, 4 figures. Accepted for publication in Ap

The Kepler-10 Planetary System Revisited by HARPS-N: A Hot Rocky World and a Solid Neptune-Mass Planet

Kepler-10b was the first rocky planet detected by the Kepler satellite and con- firmed with radial velocity follow-up observations from Keck-HIRES. The mass of the planet was measured with a precision of around 30%, which was insufficient to constrain models of its internal structure and composition in detail. In addition to Kepler-10b, a second planet transiting the same star with a period of 45 days was sta- tistically validated, but the radial velocities were only good enough to set an upper limit of 20 Mearth for the mass of Kepler-10c. To improve the precision on the mass for planet b, the HARPS-N Collaboration decided to observe Kepler-10 intensively with the HARPS-N spectrograph on the Telescopio Nazionale Galileo on La Palma. In to- tal, 148 high-quality radial-velocity measurements were obtained over two observing seasons. These new data allow us to improve the precision of the mass determina- tion for Kepler-10b to 15%. With a mass of 3.33 +/- 0.49 Mearth and an updated radius of 1.47 +0.03 -0.02 Rearth, Kepler-10b has a density of 5.8 +/- 0.8 g cm-3, very close to the value -0.02 predicted by models with the same internal structure and composition as the Earth. We were also able to determine a mass for the 45-day period planet Kepler-10c, with an even better precision of 11%. With a mass of 17.2 +/- 1.9 Mearth and radius of 2.35 +0.09 -0.04 Rearth, -0.04 Kepler-10c has a density of 7.1 +/- 1.0 g cm-3. Kepler-10c appears to be the first strong evidence of a class of more massive solid planets with longer orbital periods.Comment: 44 pages, 8 figures, accepted for publication in Ap

Autonomous drone interception with Deep Reinforcement Learning

co-located with the the 31st International Joint Conference on Artificial Intelligence and the 25th European Conference on Artificial Intelligence (IJCAI-ECAI 2022)International audienceDriven by recent successes in artificial intelligence, new autonomous navigation systems are emergingin the urban space. The adoption of such systems raises questions about certification criteria and theirvulnerability to external threats. This work focuses on the automated anti-collision systems designed forautonomous drones evolving in an urban context, less controlled than the conventional airspace andmore vulnerable to potential intruders. In particular, we highlight the vulnerabilities of such systems tohijacking, taking as example the scenario of an autonomous delivery drone diverted from its mission bya malicious agent. We demonstrate the possibility of training Reinforcement Learning agents to deflecta drone equipped with an automated anti-collision system. Our contribution is threefold. Firstly, weillustrate the security vulnerabilities of these systems. Secondly, we demonstrate the effectiveness ofReinforcement Learning for automatic detection of security flaws. Thirdly, we provide the communitywith an original benchmark based on an industrial use case

Autonomous drone interception with Deep Reinforcement Learning

co-located with the the 31st International Joint Conference on Artificial Intelligence and the 25th European Conference on Artificial Intelligence (IJCAI-ECAI 2022)International audienceDriven by recent successes in artificial intelligence, new autonomous navigation systems are emergingin the urban space. The adoption of such systems raises questions about certification criteria and theirvulnerability to external threats. This work focuses on the automated anti-collision systems designed forautonomous drones evolving in an urban context, less controlled than the conventional airspace andmore vulnerable to potential intruders. In particular, we highlight the vulnerabilities of such systems tohijacking, taking as example the scenario of an autonomous delivery drone diverted from its mission bya malicious agent. We demonstrate the possibility of training Reinforcement Learning agents to deflecta drone equipped with an automated anti-collision system. Our contribution is threefold. Firstly, weillustrate the security vulnerabilities of these systems. Secondly, we demonstrate the effectiveness ofReinforcement Learning for automatic detection of security flaws. Thirdly, we provide the communitywith an original benchmark based on an industrial use case

Micromechanical modelling of magnesia/spinel refractories: how to account of the effect of thermal history on mechanical behaviour

International audienc

Optimization of bandpass optical filters based on TiO2 nanolayers

International audienceThe design and realization of high-quality bandpass optical filters are often very difficult tasks due to the strong correlation of the optical index of dielectric thin films to their final thickness, as observed in many industrial deposition processes. We report on the optimization of complex optical filters in the visible and NIR spectral ranges as realized by ion beam-assisted electron beam deposition of silica and titanium oxide multilayers. We show that this process always leads to amorphous films prior to thermal annealing. On the contrary, the optical dispersion of TiO2 nanolayers is highly dependent on their thickness, while this dependence vanishes for layers thicker than 100 nm. We demonstrate that accounting for this nonlinear dependence of the optical index is both very important and necessary in order to obtain high-quality optical filters

Prendre au sérieux la société de la connaissance: Livre blanc

il s'agit d'un type de produit dont les métadonnées ne correspondent pas aux métadonnées attendues dans les autres types de produit : REPORTCe Livre Blanc est le fruit de quatre années de rencontres, de séminaires préparatoires, de colloques, d’auditions, de focus groups.Plus de 1.500 personnes y ont contribué depuis 2012. Il est représentatif de ce que l’on nomme « la société de la connaissance par le bas ». Il se justifie d’autant plus que la confiance de tous nos interlocuteurs dans l’apport des démarches scientifiques comme de la formation supérieure dans la construction de notre futur commun est avérée.Ce Livre Blanc est une contribution à l’analyse. Il propose des pistes d’amélioration des politiques publiques. Il vise à définir l’horizond’actions pour les acteurs concernés : le législateur, l’exécutif, lesacteurs de la société civile, les établissements d’enseignement supérieuret de recherche, et les collectivités. (...)Evelyne Lhoste et Pierre-Benoit Joly du Laboratoire Interdisciplinaire Sciences Innovations Sociétés de l'Inra font partie du comité de pilotage et de rédaction